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ABSTRACT 



A wireless handset phone, virtual image display coupled to 
the wireless handset phone and pointing device for pointing 
on the virtual image display are included. The communicator 
may also include a limited-function computer. The virtual 
image display uses optics to create the virtual image. The 
pointing device may include a virtual selector; that is, an 
array of sensors or switches to accommodate different 
users's hands. 

11 Claims, 16 Drawing Sheets 
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PERSONAL COMMUNICATOR INCLUDING 

A HANDSET PHONE WITH AN 
INTEGRATED VHITUAL IMAGE DISPLAY 

This application is a continuatioa of U^. application Ser. 
No. 08/689,546, filed Aug. 9, 1996, which claims the benefit 
of U.S. Provisional Application No. 60/004,218, filed Sep. 
21, 1995. 

BACKGROUND OF THE INVENTION 

1. Technical Field 

Hie present invention generally relates to communication 
devices. More particularly, the present invention relates to 
personal communication devices including both audio and a 
display. 

2. Background Information 

A major market (predicted to be over $1.5 billion dollars 
annually by 2000) is beginning to develop for compact 
devices combining cellular (or other wireless) telephones 
with a limited-function computer. An early example of this 
class of devices being called "personal communicators" is 
the IBM/Bell-South "SIMON". In its present form, the 
SIMON adds a relatively small computer (8086-class, mov- 
ing to 80386-class), plus a backlit, touch-sensitive display to 
a cellular telephone. This combination of computer, wireless 
communication, and enabling software creates a powerful 
productivity tool which allows a single device to provide 
wireless telephony functions, personal information manage- 
ment (e.g., electronic calendar and address/phone number 
listings), as well as two-way wireless transfer of digital data. 

Some of the limitations of these devices have heretofore 
included the difficulty of viewing the data display while 
communicating over the phone, the weight and fragility of 
the relatively large LCD display, the relatively high power 
required for the back-lit monochrome display, and the even 
higher cost and power to provide a color display. 

Furthermore, the touch- and stylus-based user interface 
generally requires two hands to operate and precludes the 
simultaneous use of the phone handset and the computer 
functions, except in a speaker-phone mode where the user 
would have to speak to the unit while holding it in front of 
him. While speech-based operation will be added to these 
devices in the future, a practical speech-based user interface 
still requires the user to see a display during use. The reason 
is that for speech navigation, the user needs to see the set of 
allowable selections (the menu items), while for dictation 
mode, the user would like to see the progress of the 
speech-to-text translation. 

As an alternative to SIMON-like designs which make it 
difficult to utilize the phone handset in the normal, private 
(non-speakerphone) mode while also viewing the display, it 
has been suggested that the personal communicator should 
be fashioned like a pair of glasses or goggles, with a 
view-finder like "projection" display and a speaker and 
microphone all built into the "frames" of the headset. While 
this interaction paradigm may eventually become common 
and widely-accepted (probably by today's video-game 
generation), it is not as natural or convenient for the user 
who frequently needs to make a quick phone call or needs 
to just briefly check some e-mail. 

Another possible alternative is to make a SIMON-like 
device where the display can be detached from the unit and 
held in the front of the user by the hand opposite the one 
holding the phone. An infrared or other wireless link 
between the phone and the display would allow data coming 
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over the main handset to be relayed to the display. Although 
this design is docs allow simultaneous viewing^taJking, it 
requires two hands for simple operation and makes pointing 
or selecting an item horn the display a difficult (possibly 
5 three-handed) operation. 

Thus, a need exists for an improved personal commimi- 
cation device including both audio and a display. 

SUMMARY OF THE INVENTION 

Briefly, the preseot invention satisfies the need for an 
improved personal communicator by providing a handsel 
phone with an integrated virtual image display, a means for 
pointing and means for selecting on the virtual image. 

In accordance with the above, it is an object of the present 
invention to provide a personal communicator with both 
audio and visual capabilities. 

It is a further object of the present invention to provide a 
personal communicator with a phone function and virtual 
2Q image display. 

It is still another object of the preseot invention to provide 
a personal communicator with an integrated pointing and 
selecting capability. 

The present invention provides a personal communicator, 
25 comprising a handset phone, a virtual image display coupled 
to the handset phone, means for pointing and means for 
selecting on the virtual image display. The personal com- 
municator may further comprise a computer, and/or a 
gravity-responsive switch for rotating the virtual image 
30 based on right-handed or left-handed use of the communi- 
cator. 

These, and other objects, features and advantages of this 
invention will become apparent from the following detailed 
description of the various aspects of the invention taken in 
35 conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 depicts a first embodiment of a personal commu- 
nicator in accordance with the present invention. 
^ FIG. 2 is a side view of the personal communicator of 
FIG. 1. 

FIG. 3 is a top view of the personal communicator of FIG. 
1 with the virtual image display rotated. 
45 FIG. 4 depicts the personal communicator of FIG. 4 in 
use. 

FIG. 5 depicts a modified version of the personal com- 
municator of FIG. 4. 

FIG. 6 depicts a second embodiment of a personal com- 
50 municator in accordance with the present invention, shown 
in an opened position. 

FIG. 7 is a side view of the personal communicator of 
FIG. 6. 

FIG. 8 depicts the personal communicator of FIG. 6 in a 
closed position. 

FIG. 9 depicts a third embodiment of a personal commu- 
nicator in accordance with the present invention, shown in 
a closed position. 

FIG. 10 depicts the personal communicator of FIG. 9 ia 
an opened position. 

FIG. 11 depicts the personal communicator of FIG. 10 
with an optional feature for the virtual display. 

FIG. 12 depicts the personal communicator of FIGS. 9 
65 through 11 in modular form. 

FIG. 13 is a block diagram/cross-sectional view of the 
modules of the personal communicator of FIG. 12. 
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FIG. 14 depicts a fourth embodiment of a personal 
commimicator in accordance with the present invention, 
shown both in an opened and closed position. 

FIG. IS is a side view of the pecsonal communicator of 
FIG. 14. 

FIG. 16 depicts a modified version of the personal com- 
municator of FIG. 14. 

BEST MODE FOR CARRYING OUT THE 
INVENTION 

The personal communicator of the present invention can 
assume many different embodiments, as wiU become clear 
from the following description. However, each embodiment 
disclosed herein includes, in some fashion, a handset phone, 
a virtual image display, means for pointing and a means for 
selecting on the virtual image display. Some general com- 
ments regarding all of the embodiments will first be given. 

The virtual image preferably looks more or less the same 
(subtending the same visual angle and potentially having the 
same resolution) as a 10-inch notebook display or a 14-inch 
display on a desktop. Some people have one strongly 
dominant eye, so they might have to use the pecsonal 
communicator on a particular side of their head while others 
would be able to transfer the phone from one ear to the other, 
as long as the display properly rotates. Preferably, when the 
display is rotated between left-side and right-side positions, 
a switch (which may be, for example, gravity operated) is 
activated that automatically flips the image on the display to 
maintain the proper orientation. 

For pointing and menu selection tasks, the handset phone 
also incorporates a pointing device, which could be 
positioned, for example, where the users thumb would 
naturally rest. Alternately, it might be positioned at a com- 
fortable position for operation by some other finger. 

Of course, to enable the actual selection of an item which 
is under the cursor (or otherwise highUghted), there is also 
included one or more "mouse" selection buttons or other 
selectors which can be used in conjunction with the pointing 
device. Although two fixed selection buttons on the handset 
would form a functional solution, it is also possible to 
provide "virtual" selectors that do not require the user to 
precisely position the fingers to specific gripping positions 
by covering at least one edge of the handset portion of the 
personal communicator with a relatively high-resolution 
array of pressure sensors, small switches, or the like. If the 
spacing between the sensors/switches is smaller than a given 
average finger width, then it is possible for an integrated 
computer (described more fiilly with reference to FIG. 13) 
to sense the location of the user's different fingprs. With this 
technique, it is feasible for a select button function to be 
determined by the finger which is used to operate it rather 
than by a particular switch at a fixed location. In other 
words, the communicator can allow the left mouse button 
function to be initiated by, for example, a brief squeeze of 
the user's index finger, regardless of where that index finger 
is positioned. Similarly, the right mouse button function 
might be initiated by a squeeze of the user's second finger, 
again independent of the precise positioning of the fingers. 
Alternatively, these could be separate virtual selectors (e.g., 
two rows of sensors/switches) for the selection function, 
operable by the same finger or different fingers. 

FIG. 1 depicts personal communicator 10 comprising 
handset phone 12 and virtual image display 14. Handset 
phone 12 comprises speaker 16, keypad 18 and microphone 
20. Virtual image display 14 comprises curved mirror 22, 
partially refiective/transmissive optical element 24, display 
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26, rotating base 28, and a microprocessor or other inte- 
grated computer, (see FIG. 13). In addition, a stick-type 
pointing device 30 and selector 32 (i.e., one example of a 
means for pointing) provide control for a cursor viewed by 
5 a user on the virtual image display. Preferably, selector 32 
corresponds to another pointing device on the opposite side 
34 so that a uscr^s thumb can control the pointing device, 
while their index or other finger controls the selector. The 
pointing means may provide, for example, one-dimensional 
or two-dimensional navigation of a cursor or other high- 
lighter. 

Wiih presendy available technology, it is also feasible to 
add large-vocabulary speech recognition as the pointing 
and/or selecting means, although this would likely affect the 

J 5 cost and power dissipation of the commimicator. As one 
skilled in the art of speech recognition will know, speech 
recognition could be implemoited in hardware alone, a 
combination of hardware and software, or, preferably, soft- 
ware alone. One example of a combination speech recog- 

2Q nition implementation is IBM's VoiceType® Dictation 2.0. 
One example of a software speech recognition implemen- 
tation is IBM's VoiceType® Application Factory, which 
runs on at least a 486-class processor, and provides both 
command and control functions. Another example of a 

25 software implementation is IBM's VoiceType® Dictation 
3.0, whidi runs on at least a Pentium®-class processor, and 
provides command and control functions, as well as dicta- 
tion. 

Display 26 is preferably small, on the order of 33 mm or 

30 less diagonally, and can be monochrome or color. Partially 
refiective/transmissive optical element 24 could take several 
different forms, but the purpose is to reflect light from 
display 26 to ciu-vcd mirror 22, while passing light from 
curved mirror 22. Partially refiective/transmissive optical 

35 element 24 could, for example, be a beamsplitter, and may 
incorporate a polarizer. Curved mirror 22 serves the purpose 
of reflecting the image from display 26 through partially 
refiective/transmissive optical element 24, creating a virtual 
image viewable while personal communicator 10 is being 

40 held to the ear of a user for optional simultaneous use of 
handset phone 12. Curved mirror 22 could, for example, 
have a radius of curvature on the order of twice the path 
length of the image &om display 26 thereto (plus or minus 
approximately 20%). 

45 Handset phone 12 and virtual image display 14 could be 
used together to enable wireless computer functions. For 
example, the handset phone could receive (and optionally 
transmit) digital data for the integrated computer, which 
displays information in the form of a virtual image. 

50 Alternatively, the integrated computer could be stand alone, 
or receive (and optionally transmit) digital data separate 
&om the phone handset. This would allow a user to speak on 
the handset phone vdiile using the integrated computer. Still 
another option is for the virtual image to be that of the person 

55 the user is speaking to on the handset phone. 

In practice, the location of the virnial image is adjusted by 
rotating virtual image display 14 on rotating base 28. 
Further, a cursor element in the virtual image is controlled 
by a stick-type pointing device 30 and corresponding selec- 

60 tor 132 (on side 34 opposite selector 32 — see FIG. 3) 
conveniently located on handset phone 12 for a user's thumb 
and index finger, respectively. In addition, a second pointing 
device 130 (see FIG. 3) corresponding to selector 32 is 
located on the opposite side 34 of handset phone 12. Further, 

65 handset phone 12 may be a wireless handset phone (e.g. 
analog or digital cellular phone, or radio-signal based). Still 
further, personal communicator 10 may include a gravity- 
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responsive switch (see FIG. 13). Depending on whether diffractive or holographic elements tend to be sensitive (or 

personal communicator 10 is used in the left or right hand of behave dififerently) to wavelength vaiiatioas. Also shown as 

a user» the gravity -responsive switch would rotate the virtual part of virtual image display 42 are curved mirror 48, 

image 180*^ to accommodate same. partially reflective/transmissive optical element 50, and 

FIG. 2 is a side view of personal communicator 10 from ^ ambient light bloddng optical element 52. 

FIG. 1, along with an optional blocking optical element 36. FIG. 7 is a side view of the personal communicator of 

Element 36 helps prevent glare on the virtual image from FIG. 6. As shown in FIG. 7, light from display 44 reflects off 

surrounding light. The creation of the virtual image will now partially reflective/transmissive optical element 50 and onto 

be described in detail. As a real image is produced on display curved mirror 48. Curved mirror 48 then reflects the light 

26, the light from display 26 travels to partially reflective/ lO back to partially reflective/transmissive optical element 50, 

transmissive optical element 24, where some portion of it is which passes therethrough to the second optical element 46, 

reflected thereby to curved mirror 22, where it is reflected which redirects the light for viewing from at least two 

back to partially reflective/transmissive optical element 24. different directions. Again, blocking surface 52 serves to 

Since partiaUy reflective/transmissive optical element 24 prevent ambient light from interfering with the virtual 

aUows some of the light originating from display 26 to pass 15 image. 

therethrough upon reflection off curved mirror 22, a virtual piG. 8 depicts another difference between the second and 

image is thereby created, viewable by a user. Optional fijst embodiments. Shown in FIG. 8 are the various optical 

blocking element 36 may not be necessary where the par- elements of virtual image display 42 in a closed position, 

liaUy reflective/transmissive optical element 24 incorporates whereby the optical elements are foldable. This could be 

a polarizer. 20 accomplished, for example, by utilizing hinge members in 

FIG. 3 is a top view of the personal conununicator 10 of an area 51 joining curved mirror 48 and blocking surface 52, 

FIG. 1 with the virtual image display 14 rotated on base 28. and in an area 53 joining curved min'or-48 and base 55. 

In addition, from the top view, the other set of pointing FIG. 9 depicts a third embodiment of a personal commu- 

device 130 and selector 132 are shown. As described above, nicator 54 in accordance with the present invention. Handset 

pointing device 130 preferably corresponds to selector 32, ^ phone 56 is similar to the handset phones for the other 

and pointing device 30 preferably corresponds to selector embodiments. However, virtual image display 58 is different 

132. in design from the other embodiments, ^^tual image display 

FIG. 4 depicts the personal communicator of FIG. 3 in 58 comprises display 60 and curved mirror 62 (shown in a 

use. As shown, a user's thumb 134 operates stick*type ^ closed position). As can be seen, curved mirror 62 is longer 

pointing device 30, while the correspondLig selector 132 is than the curved mirror of the previous embodiments, 

operated by the user's index finger 136. Further, curved mirror 62 is attached to base 64 by a pair of 

HG. 5 depicts a personal communicator 138 that is a support arms (e.g., support arm 66). The support arms are 

modified version of personal communicator 10 shown in attached to base 64 by ball joints or the like (e.g., ball joint 

FIG. 4. The only difference between communicator 138 and 35 ^^)* avowing curved mirror 62 to assume the closed position 

communicator 10 is that communicator 138 replaces the shown in FIG. 9. 

individual selectors 32 and 132 with virtual selectors 140 FIG. 10 depicts the personal communicator 54 of FIG. 9 

and 142, respectively. The virtual selectors are each com- in the opened position. Unlike the first two embodiments, 

prised of a plurality of individual pressure sensors, switches the embodiment depicted in FIGS. 9 and 10 does not have 

or the like (e.g., sensor 144). As previously described, such 40 a partially reflective/transmissive optical element. Rather, 

virtual selectors would allow selection to be accomplished, curved mirror 62 is sized and spaced from display 62 to 

regardless of the actual location of the user's finger. Thus, provide a virtual image from a single reflection. Optionally, 

any of the user's fingers (e.g., index finger 136, middle pivots, ball joints or the like could be included at the 

finger 146. or third finger 148) could be used to perform the connection of the support arms and curved mirror 62 (e.g., 

selection task. 45 at location 70). This would allow curved mirror 62 to be 

FIG. 6 depicts a second embodiment of a personal com- ^^^^^ ^ direction 72 toward display 60 and/or in a 

municator 38 in accordance with the present invention. The direction 74 away therefrom. 

handset phone portion 40 of personal communicator 38 is FIG. 11 depicts the personal communicator 54 of FIG. 10 

similar to that in the first embodiment However, virtual with optional parallctogram linkage motion for curved mir- 

image display 42, while operable according to the same 50 ror 62. With appropriate jointing at either end of the support 

basic principles, is somewhat different than that in the first anns (e.g., support arm 66), curved mirror 62 could be 

embodiment. For example, display 44 of virtual image moved in a direction 76 to the right of the original center 

display 42 is fixed and does not rotate. As another example, position shown in FIG. 10, or in a direction 78 to the left of 

virtual image display 42 includes another optical element 46 the original center position (shown in phantom in FIG. 

allowing the virtual image to be viewed during left-hand or 55 FIG. 12 depicts the personal communicator 54 of FIGS. 9 

right-hand use of personal communicator 38 without manu- through 11 in modular fonm. Handset phone 56 and battery 

aUy rotating the virtual image display, as done in the first pack 82 could comprise, for example, a conventional cellu- 

embodiment. Optical element 46 may be, for example, a laror other wireless phone. Base 64 houses display 60 along 

diffraction grating or a lenticular lens. As one skilled in the with a limited function or other computer (see FIG. 13). 

art wiU know, a diffraction grating is a transparent plate or 60 However, it would be understood that battery pack 82, if 

member including many finely ruled, closely spaced, usually designed only for a phone, would have a more limited 

equidistant horizontal lines, but may also be made holo- operational period when tised in communicator 54, due to 

graphically consisting of, for example, a di-chromated gela- the increased power needs from, e.g., display 60. In addition, 

tin layer sandwiched between glass plates; and a lenticular such a modular design would allow one to take only the 

lens is a saw-toothed, grooved structure splitting the image 6S handset phone portion 56 (and battery 82) on particular 

into at least two directions. The inclusion of optical element occasions, rather than all the modules of personal commu- 

46 may require that display 44 be monochrome, since nicator 54. 
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As used in connection with the present exemplary 
embodiment, the term "limited-function computer" refers to 
a computer that would coordinate the functions of the 
handset phone and virtual image display, as well as allow a 
pointing device and selector to be used with the virtual ^ 
image display. As one example, a microprocessor could be 
used. It wiW be understood that a computer with greater or 
different functionality could be used in the present invention. 

FIG. 13 is a block diagram/cross-sectional view of the 
various modtiles of personal communicator 54 from FIG. 12 
(shown without the curved mirror and support arms). A 
connector 84 from base 64 mates with a connector 88 on 
handset phone 56, providing a connection between handset 
phone 56 and computer 86. In addition, power connections 
are made from handset phone 56 through base 64 to battery 
pack 82 (e.g., corresponding power connections 90, 92, 94 15 
and 96). Optionally, the power on and/or off for handset 
phone 56 may be separate from that for base 64. As another 
option, display 60 and/or computer 86 may include auto- 
matic shut-down when not in use, similar to modem note- 
book computers. 20 

Handset phone 56 comprises microphone 150, keypad 
152, speaker 154, radio 156, antenna 158, micro-controller 
160 and data/voice link 162 between connector 88 and 
micro-controller 160. One skilled in the art will understand 
the operation of handset phone 56. 25 

Base 64 comprises display 60, computer 86 and gravity 
switch 164. Computer 86 comprises micro-processor 166, 
DRAM 168 and flash storage 170. A data/voice I/O interface 
172 connects computer 86 via dataA^oice link 174 to micro- 
phone 150. A display interface 176 connects display 60 with 3Q 
computer 86 and the pointing means (through interface 178). 
The pointing means comprises pointers 180 and 182, along 
with the corresponding selectors 184 and 186, respectively. 
Optionally, another dataA^oice link 188 oould be included for 
connecting to external devices, such as, for example, a 
display, a microphone, a keyboard, or other I/O devices and 
combinations thereof. Further, optional devices 190 may be 
included and linked to bus 192. For example, storage 
expansion, a security smartcard or bar code or magnetic 
stripe reader could be included. 

FIG. 14 depicts a fourth embodiment of a personal ^ 
commtmicator 98 in accordance with the present invention. 
Personal commtmicator 98 is similar to the third embodi- 
ment shown in FIG. 10, however, the means of support for 
curved mirror 100 are different. Specifically, three semi- 
flexible straps 102, 104 and 106 connect curved mirror 100 45 
to base 108. As shown in FIG. 14, semi-flexible strap 102 is 
generally obscured by strap 106. Straps 102 and 104 are 
below strap 106. Semi-flexible straps 102, 104 and 106, in 
their most crude form, could be, is for example, similar to a 
metal tape measure. As such, the straps are rigid when in the 
open position, but are flexible enough to bend such that a 
closed position can be assumed (FIG. 14 also shows the 
closed position). Locations 110 and 112 are possible loca- 
tions for some type of closure mechanism (with or without 
a separate release button) for the curved mirror. 

FIG. 15 is a side view of the personal communicator 98 
of FIG. 14. Optionally, curved mirror 100 may be connected 
to the semi-flexible straps by, for example, baU joints (e.g., 
ball joint 114). Such a connection would allow curved mirror 
100 to be titled in a direction 116 toward base 108 or in a 
direction 118 away from base 108, if the mounting at 
location 120 allows for at least limited sliding motion for 
strap 106. In addition, a swivel joint at location 120 would 
allow curved mirror 100 to be rotated to left or right of the 
center position shown in FIG. 14. 

In an altemative version of the fourth embodiment, shown 65 
in FIG. 16, bottom straps 102 and 104 are replaced by a 
single semi-flexible strap 122 connected to curved mirror 



100 by a ball joint 124. Strap 106 above strap 122 is also 
connected to curved mirror 100 by a ball joint 126. With ball 
joints 124 and 126 in a vertical line, curved mirror 100 could 
partially pivot about a vertical axis 128 to permit viewing in 
either right- or left-handed usage. 

While several aspects of the present invention have been 
described and depicted herein, alternative aspects may be 
effeaed by those skilled in the art to accomplish the same 
objectives. Accordingly, it is intended by the appended 
claims to cover all such altemative aspects as fall within the 
true spirit and scope of the invention. 

What is claimed is: 

1. A personal communicator, comprising: 
a handset phone; 

a virtual image display coupled to the handset phone; 
means coupled to the handset phone for positioning a 

cursor on the virtual image display; and 
means for selecting on the virtual image display; 
wherein the handset phone comprises a gravity- 
responsive switch for rotating an image on the virtual 
image display based on right-handed or left-handed 
use of the handset phone. 

2. A personal communicator, comprising: 
a handset phone; 

a virtual image display coupled to the handset phone; 
means coupled to the handset phone for positioning a 

cursor on the virtual image display; and 
means for selecting on the virtual image display; 

wherein the orientation of an image on the virtual 
image display is adjustable. 

3. A personal communicator, comprising: 
a handset phone; 

a virtual image display coupled to the handset phone; 
means coupled to the handset phone for positioning a 

cursor on the virtual image display; and 
means for selecting on the virtual image display; 
wherein the virtual image display comprises: 

a curved mirror; and 

a partially reflective/transmissive optical element. 

4. The personal communicator of claim 3, wherein the 
partially reflective/transmissive optical element comprises a 
polarizer. 

5. The personal commtmicator of claim 3, wherein the 
partially reflective/transmissive optical element comprises a 
beamsplitter. 

6. The personal communicator of claim 3, further com- 
prising another optical element for redirecting light such that 
a virtual image from the virtual image display may be 
viewed from at least two different directions. 

7. The personal communicator of claim 6, wherein the 
another optical element comprises a lenticular lens. 

8. The personal communicator of claim 6, wherein the 
another optical element comprises a diffraction grating. 

9. The personal communicator of claim 3, further com- 
prising a blocking optical element. 

10. A personal commtmicator, comprising: 
a handset phone; 

a virtual image display coupled to the handset phone; 
means coupled to the handsel phone for positioning a 

cursor on the virtual image display; and 
means for selecting on the virtual image display; 

wherein the virtual image display comprises a curved 
mirror. 

11. The personal communicator of claim 10, wherein the 
curved mirror is coupled to the handset phone by a support. 
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